The maximum entropy ansatz in the absence of a time arrow: fractional pole models

The maximum entropy ansatz, as it is often invoked in the context of time-series analysis, suggests the selection of a power spectrum which is consistent with autocorrelation data and corresponds to a random process least predictable from past observations. We introduce and compare a class of spectra with the property that the underlying random process is least predictable at any given point from the complete set of past and future observations. In this context, randomness is quantified by the size of the corresponding smoothing error and deterministic processes are characterized by integrability of the inverse of their power spectral densities--as opposed to the log-integrability in the classical setting. The power spectrum which is consistent with a partial autocorrelation sequence and corresponds to the most random process in this new sense, is no longer rational but generated by finitely many fractional-poles.Comment: 18 pages, 3 figure

Relative entropy and the multi-variable multi-dimensional moment problem

Entropy-like functionals on operator algebras have been studied since the pioneering work of von Neumann, Umegaki, Lindblad, and Lieb. The most well-known are the von Neumann entropy $trace (\rho\log \rho)$ and a generalization of the Kullback-Leibler distance $trace (\rho \log \rho - \rho \log \sigma)$, refered to as quantum relative entropy and used to quantify distance between states of a quantum system. The purpose of this paper is to explore these as regularizing functionals in seeking solutions to multi-variable and multi-dimensional moment problems. It will be shown that extrema can be effectively constructed via a suitable homotopy. The homotopy approach leads naturally to a further generalization and a description of all the solutions to such moment problems. This is accomplished by a renormalization of a Riemannian metric induced by entropy functionals. As an application we discuss the inverse problem of describing power spectra which are consistent with second-order statistics, which has been the main motivation behind the present work.Comment: 24 pages, 3 figure

An intrinsic metric for power spectral density functions

We present an intrinsic metric that quantifies distances between power spectral density functions. The metric was derived by the author in a recent arXiv-report (math.OC/0607026) as the geodesic distance between spectral density functions with respect to a particular pseudo-Riemannian metric motivated by a quadratic prediction problem. We provide an independent verification of the metric inequality and discuss certain key properties of the induced topology.Comment: 7 page

Distances between power spectral densities

We present several natural notions of distance between spectral density functions of (discrete-time) random processes. They are motivated by certain filtering problems. First we quantify the degradation of performance of a predictor which is designed for a particular spectral density function and then it is used to predict the values of a random process having a different spectral density. The logarithm of the ratio between the variance of the error, over the corresponding minimal (optimal) variance, produces a measure of distance between the two power spectra with several desirable properties. Analogous quantities based on smoothing problems produce alternative distances and suggest a class of measures based on fractions of generalized means of ratios of power spectral densities. These distance measures endow the manifold of spectral density functions with a (pseudo) Riemannian metric. We pursue one of the possible options for a distance measure, characterize the relevant geodesics, and compute corresponding distances.Comment: 16 pages, 4 figures; revision (July 29, 2006) includes two added section

The Separation Principle in Stochastic Control, Redux

Over the last 50 years a steady stream of accounts have been written on the separation principle of stochastic control. Even in the context of the linear-quadratic regulator in continuous time with Gaussian white noise, subtle difficulties arise, unexpected by many, that are often overlooked. In this paper we propose a new framework for establishing the separation principle. This approach takes the viewpoint that stochastic systems are well-defined maps between sample paths rather than stochastic processes per se and allows us to extend the separation principle to systems driven by martingales with possible jumps. While the approach is more in line with "real-life" engineering thinking where signals travel around the feedback loop, it is unconventional from a probabilistic point of view in that control laws for which the feedback equations are satisfied almost surely, and not deterministically for every sample path, are excluded.Comment: 23 pages, 6 figures, 2nd revision: added references, correction